In-order sliding-window aggregation in worst-case constant time
نویسندگان
چکیده
Sliding-window aggregation is a widely-used approach for extracting insights from the most recent portion of data stream. While aggregations interest can usually be expressed as binary operators that are associative, they not necessarily commutative nor invertible. Non-invertible operators, however, difficult to support efficiently. DABA first algorithm sliding-window with worst-case constant time. Prior DABA, best published algorithms would require $$O(\log n)$$ steps per window operation size n—and while strictly in-order streams, this bound could improved O(1) in amortized sense, it was known how achieve an worst case, which critical latency-sensitive applications. In article, besides describing more detail, we introduce new variant, Lite, achieves same time bounds less memory. Whereas requires space storing 2n partial aggregates, Lite only $$n+2$$ aggregates. Our experiments on synthetic and real theoretical findings.
منابع مشابه
Constant-Time Sliding Window Aggregation
Sliding-window aggregation is a widely-used approach for extracting insights from the most recent portion of a data stream. Most aggregation operations of interest can be cast as binary operators that are associative, but not necessarily commutative nor invertible. However, non-invertible operators are nontrivial to support efficiently. The best existing algorithms for this setting require Oplo...
متن کاملDynamic Dictionaries in Constant Worst-Case Time
We introduce a technique to maintain a set of n elements from a universe of size u with membership and indel operations, so that elements are associated r-bit satellite data. We achieve constant worst-case time for all the operations, at the price of spending u + o(u) + O(nr + n log log log u) bits of space. Only the variant where the space is of the form O(nr + n log u) was exhaustively explor...
متن کاملGeneral Incremental Sliding-Window Aggregation
Stream processing is gaining importance as more data becomes available in the form of continuous streams and companies compete to promptly extract insights from them. In such applications, sliding-window aggregation is a central operator, and incremental aggregation helps avoid the performance penalty of re-aggregating from scratch for each window change. This paper presents Reactive Aggregator...
متن کاملPurely Functional Worst Case Constant Time Catenable Sorted Lists
We present a purely functional implementation of search trees that requires O(log n) time for search and update operations and supports the join of two trees in worst case constant time. Hence, we solve an open problem posed by Kaplan and Tarjan as to whether it is possible to envisage a data structure supporting simultaneously the join operation in O(1) time and the search and update operation...
متن کاملFDiBC: A Novel Fraud Detection Method in Bank Club based on Sliding Time and Scores Window
One of the recent strategies for increasing the customer’s loyalty in banking industry is the use of customers’ club system. In this system, customers receive scores on the basis of financial and club activities they are performing, and due to the achieved points, they get credits from the bank. In addition, by the advent of new technologies, fraud is growing in banking domain as well. Therefor...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: The Vldb Journal
سال: 2021
ISSN: ['0949-877X', '1066-8888']
DOI: https://doi.org/10.1007/s00778-021-00668-3